Updating DXL attributes in multiple modules

Hi,

Please can someone guide me on the best way to update DXL attributes in multiple modules in one go?

I have a DXL attribute which returns the last signed baseline for the module. The DXL attribute is in every module but now I want to update the code but I have about 30 modules to update.

Thanks,

Glyn
SystemAdmin - Wed Jan 11 05:55:40 EST 2012

Re: Updating DXL attributes in multiple modules
rain_bow - Wed Jan 11 08:14:26 EST 2012

For just 30 modules, it probably will be faster to do the task manually.

Re: Updating DXL attributes in multiple modules
BillTidy - Wed Jan 11 08:42:40 EST 2012

rain_bow - Wed Jan 11 08:14:26 EST 2012
For just 30 modules, it probably will be faster to do the task manually.

Never put code into your modules - it is a headache to maintain. Write the code and put it in an include file and then in your module dxl attribute just call the inc file. You only then maintain code in one place (in the file on disk) and whenever you need to change it the code is updated immediately in all modules referencing it.

Re: Updating DXL attributes in multiple modules
SystemAdmin - Wed Jan 11 11:15:07 EST 2012

BillTidy - Wed Jan 11 08:42:40 EST 2012
Never put code into your modules - it is a headache to maintain. Write the code and put it in an include file and then in your module dxl attribute just call the inc file. You only then maintain code in one place (in the file on disk) and whenever you need to change it the code is updated immediately in all modules referencing it.

Where do I store DXL code to use as an include that will work in everyone's client?

Re: Updating DXL attributes in multiple modules
llandale - Wed Jan 11 14:43:43 EST 2012

SystemAdmin - Wed Jan 11 11:15:07 EST 2012
Where do I store DXL code to use as an include that will work in everyone's client?

If your folks are all close by and you are SURE every one has access to one specific server, you can put this in your attr-DXL:
  • #include <\MyServer\MyDxl\MyAttrDXL\ThisParticularAttrDXL.dxl>
Then just update that file.

I emphasize the "If" in the 1st paragraph. In big organizations that's often not true, and someone who looks at the attr will get numerous rude DXL errors.

To "push" attributes to many modules, manually update one such module, save it, move it to some folder (perhaps the Project folder), then use script "copyAtts.dxl", of which there are lots of versions out there. Then move the module back to its permanant location. This is a reason to have "Template" modules, of a few different types (Requiremetns, Tests...), keep those up to date and just push them around to keep all your module's attributes up to date.

Having said that, I intend to do this: write a custom DXL that lists every module and every attribute and the file containing that attributes Attr-DXL, then run that on-demand script to update all my project's Attr-DXL. The main reason is because I'm now writing "context sensitive" Attr-DXL that looks at the current attribute name to determine what exactly what to do. Script "Interface_Summary.dxl" displays different sorts of summaries depending on the name of the attribute e.g. "InterfaceAttrs", "SourceAttrs", "DestinationAttrs" bla bla bla. I may have 20 Attr-DXL in a module and don't want to even do one such template module manually.

I'm considering another approach, some module-level attribute lists the attr-DXL names and the source DXL file name, and the script reads that and updates this module accordingly.

These let me have my cake and eat it too, and single repository of managed DXL available to lots of folks who may not have access to that server folder.

-Louie

Actually, since I'm a dweeb, I've proved the concept of "deploying" the code in the database in configuration files, and the Attr-DXL reads the config file, eval_ and display the results. That's too slow but the concept will work for Addins menu problems with distributed clients. The database admins determine which addins are appropriate for their database.

Re: Updating DXL attributes in multiple modules
adevicq - Fri Jan 27 08:50:45 EST 2012

llandale - Wed Jan 11 14:43:43 EST 2012
If your folks are all close by and you are SURE every one has access to one specific server, you can put this in your attr-DXL:

  • #include <\MyServer\MyDxl\MyAttrDXL\ThisParticularAttrDXL.dxl>
Then just update that file.

I emphasize the "If" in the 1st paragraph. In big organizations that's often not true, and someone who looks at the attr will get numerous rude DXL errors.

To "push" attributes to many modules, manually update one such module, save it, move it to some folder (perhaps the Project folder), then use script "copyAtts.dxl", of which there are lots of versions out there. Then move the module back to its permanant location. This is a reason to have "Template" modules, of a few different types (Requiremetns, Tests...), keep those up to date and just push them around to keep all your module's attributes up to date.

Having said that, I intend to do this: write a custom DXL that lists every module and every attribute and the file containing that attributes Attr-DXL, then run that on-demand script to update all my project's Attr-DXL. The main reason is because I'm now writing "context sensitive" Attr-DXL that looks at the current attribute name to determine what exactly what to do. Script "Interface_Summary.dxl" displays different sorts of summaries depending on the name of the attribute e.g. "InterfaceAttrs", "SourceAttrs", "DestinationAttrs" bla bla bla. I may have 20 Attr-DXL in a module and don't want to even do one such template module manually.

I'm considering another approach, some module-level attribute lists the attr-DXL names and the source DXL file name, and the script reads that and updates this module accordingly.

These let me have my cake and eat it too, and single repository of managed DXL available to lots of folks who may not have access to that server folder.

-Louie

Actually, since I'm a dweeb, I've proved the concept of "deploying" the code in the database in configuration files, and the Attr-DXL reads the config file, eval_ and display the results. That's too slow but the concept will work for Addins menu problems with distributed clients. The database admins determine which addins are appropriate for their database.

All,

You can find attached a script (zip) called "findAndReplaceDXLAttributesInProject.dxl" that allows to find DXL attributes that contain a certain string or find and replace the string with another string.
The function scans a folder and all its subfolders.
Another DXL in the szip file contains a recursive function that scans a folder / project and calls a callback function.

Feel free to use it. You can of course check the code and give me youf feedback.
I have used it many times in my company and has been very helpful.

Best regards,

Alain
Attachments

attachment_14782971_findAndReplaceInDXLAttributes.zip

Re: Updating DXL attributes in multiple modules
llandale - Fri Jan 27 10:40:57 EST 2012

adevicq - Fri Jan 27 08:50:45 EST 2012
All,

You can find attached a script (zip) called "findAndReplaceDXLAttributesInProject.dxl" that allows to find DXL attributes that contain a certain string or find and replace the string with another string.
The function scans a folder and all its subfolders.
Another DXL in the szip file contains a recursive function that scans a folder / project and calls a callback function.

Feel free to use it. You can of course check the code and give me youf feedback.
I have used it many times in my company and has been very helpful.

Best regards,

Alain

Feeback:
Very good idea.

These three lines starting at line #85
  • newValue = theDXL[0:offs-1]
  • newValue = newValue "" sReplace
  • newValue = newValue "" theDXL[(offs+length(sFind)):]
Should take advantage of the Buffer, and do this:
  • newValue = theDXL[0:offs-1]
  • newValue += sReplace
  • newValue += theDXL[(offs+length(sFind)):]
Otherwise you are wasting a lot of string memory with the unneeded concatenations

Probably doesn't matter, but I'd change the for loop starting at line #58 to this:
  • for ad in m do {
  • if ((bool ad.dxl) && ad.useraccess && canModify(ad)) {
Not sure you really WANT to restrict it to just "string" or "text" or object only attr-DXL.

I notice you only change one occurance of the string; ignoring any others.

The contains(string) function looks for "keywords", and will not find "aint" in "taint". Since you are likely looking for whole word phrases that doesn't matter, in this case.

Please provide an example of when you would want to replace a phrase (only) in all such DXL.

I'm thinking of stealing this DXL, and modifying it such that it looks for a particular string in the DXL, and if found replaces the entire code with some selected file. If I then put the name of the DXL in the dxl, perhaps "dxl_Display_Sections.dxl", then when I update that file I can replace all the attr-DXL that have that file in them. For me, I would restrict it to only look for that file name in the 1st line of the DXL since I always put it there.

-Louie

Re: Updating DXL attributes in multiple modules
llandale - Fri Jan 27 10:48:04 EST 2012

llandale - Fri Jan 27 10:40:57 EST 2012
Feeback:
Very good idea.

These three lines starting at line #85

  • newValue = theDXL[0:offs-1]
  • newValue = newValue "" sReplace
  • newValue = newValue "" theDXL[(offs+length(sFind)):]
Should take advantage of the Buffer, and do this:
  • newValue = theDXL[0:offs-1]
  • newValue += sReplace
  • newValue += theDXL[(offs+length(sFind)):]
Otherwise you are wasting a lot of string memory with the unneeded concatenations

Probably doesn't matter, but I'd change the for loop starting at line #58 to this:
  • for ad in m do {
  • if ((bool ad.dxl) && ad.useraccess && canModify(ad)) {
Not sure you really WANT to restrict it to just "string" or "text" or object only attr-DXL.

I notice you only change one occurance of the string; ignoring any others.

The contains(string) function looks for "keywords", and will not find "aint" in "taint". Since you are likely looking for whole word phrases that doesn't matter, in this case.

Please provide an example of when you would want to replace a phrase (only) in all such DXL.

I'm thinking of stealing this DXL, and modifying it such that it looks for a particular string in the DXL, and if found replaces the entire code with some selected file. If I then put the name of the DXL in the dxl, perhaps "dxl_Display_Sections.dxl", then when I update that file I can replace all the attr-DXL that have that file in them. For me, I would restrict it to only look for that file name in the 1st line of the DXL since I always put it there.

-Louie

I would "ErrMess = checkDXL(stringOf(newValue))" to make sure the DXL at least interprets.

I would see if the "modify" actually worked; look for the returned AttrDef or surround with noError()/lastError()

Your dealWithFolder/Module schema is pretty clever.

-Louie

Re: Updating DXL attributes in multiple modules
adevicq - Fri Jan 27 11:00:38 EST 2012

llandale - Fri Jan 27 10:48:04 EST 2012
I would "ErrMess = checkDXL(stringOf(newValue))" to make sure the DXL at least interprets.

I would see if the "modify" actually worked; look for the returned AttrDef or surround with noError()/lastError()

Your dealWithFolder/Module schema is pretty clever.

-Louie

Louie,

Thanks for your comment.

About concatenation instead of using "+=": I've had some issues with this in DXL (in some circumstances it seems it does not work so I abandoned it... maybe I'm wrong).

About the checkDXL: in fact if the DXL is not valid it is impossible to save the module. So I left it like this as it is impossible to introduce a script that contains syntax errors...

I wrote this script initially to change the location of include files generally located on a server but I have changed its organization or location a couple of times. therefore most of the time I had to change something like "//myserver/myfolder/.../" by "//myotherserver/myotherfolder/.../". In this case I have very few chances to replace something I'm not supposed to change!

Feel free to enhance it as you wish (and of course share those changes on this forum!)

Many thanks

Alain

Re: Updating DXL attributes in multiple modules
llandale - Fri Jan 27 13:32:38 EST 2012

adevicq - Fri Jan 27 11:00:38 EST 2012
Louie,

Thanks for your comment.

About concatenation instead of using "+=": I've had some issues with this in DXL (in some circumstances it seems it does not work so I abandoned it... maybe I'm wrong).

About the checkDXL: in fact if the DXL is not valid it is impossible to save the module. So I left it like this as it is impossible to introduce a script that contains syntax errors...

I wrote this script initially to change the location of include files generally located on a server but I have changed its organization or location a couple of times. therefore most of the time I had to change something like "//myserver/myfolder/.../" by "//myotherserver/myotherfolder/.../". In this case I have very few chances to replace something I'm not supposed to change!

Feel free to enhance it as you wish (and of course share those changes on this forum!)

Many thanks

Alain

"+=" is pretty standard and I haven't had trouble with it. For me the right side is always a string. if you can recall the trouble please let us know.

If you are reusing the buffer, the 1st insert should be "=" and subsequent ones "+=":

Buffer bufGlobal = create()
string DoSomething(()
{ bufGlobal  = 1st line    // <<-- This is the mindful step
  bufGlobal += 2nd line
  bufGlobal += 3rd line
  return(stringOf(bufGlobal))
}


That construct solves the problem of creating/deleting the Buffer inside the function which is particularly slow.

It seems to me however I have had trouble with re-using such Buffers, IIRC it was when the buffer used to be large and is now small and you try to write it to a Stream file.

I think you are better off displaying an "interpret" error from checkDXL than allowing the modify to fail; which won't tell the user why.

-Louie

Re: Updating DXL attributes in multiple modules
Mathias Mamsch - Sun Jan 29 06:32:00 EST 2012

llandale - Fri Jan 27 13:32:38 EST 2012

"+=" is pretty standard and I haven't had trouble with it. For me the right side is always a string. if you can recall the trouble please let us know.

If you are reusing the buffer, the 1st insert should be "=" and subsequent ones "+=":

Buffer bufGlobal = create()
string DoSomething(()
{ bufGlobal  = 1st line    // <<-- This is the mindful step
  bufGlobal += 2nd line
  bufGlobal += 3rd line
  return(stringOf(bufGlobal))
}


That construct solves the problem of creating/deleting the Buffer inside the function which is particularly slow.

It seems to me however I have had trouble with re-using such Buffers, IIRC it was when the buffer used to be large and is now small and you try to write it to a Stream file.

I think you are better off displaying an "interpret" error from checkDXL than allowing the modify to fail; which won't tell the user why.

-Louie

Louie, you keep saying that allocating a buffer is really slow. Just to see how slow, slow really is, I scribbled the below example. On my PC the appending of three strings to a buffer and doing stringOf is 1.5x - 2x slower than allocating and deleting a buffer (110ms vs. 187ms on DOORS 8.2). Of course it would be totally unnecessary to allocate the buffer in that function, so of course the global variable is the better option. I just wanted to point out that allocating a buffer and appending a string to a buffer nearly take the same time (at least when the allocation list is empty). Also note that both operations are very fast (90ms / 100000 = 900 ns) and most of the time it will be no improvement to the overall speed of the program to do so.

Regards, Mathias

By the way: Try putting the code into a function and see how the times change!
 

pragma runLim, 0
 
Buffer buf 
 
int i, start
 
string a = "123456"
string b = "678909"
string c = "abcdef"
 
start = getTickCount_()
for (i = 0; i < 100000; i++) {
    buf = create()
        delete buf
}
print "Allocating and Deleting Buffer:" (getTickCount_() - start) "\n"
 
 
buf = create()
start = getTickCount_()
for (i = 0; i < 100000; i++) {
        buf = a 
        buf += b
        buf += c
        stringOf buf
}
print "Appending Stuff To Buffer:" (getTickCount_() - start) "\n"


 


Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

 

Re: Updating DXL attributes in multiple modules
llandale - Mon Jan 30 19:01:42 EST 2012

Mathias Mamsch - Sun Jan 29 06:32:00 EST 2012

Louie, you keep saying that allocating a buffer is really slow. Just to see how slow, slow really is, I scribbled the below example. On my PC the appending of three strings to a buffer and doing stringOf is 1.5x - 2x slower than allocating and deleting a buffer (110ms vs. 187ms on DOORS 8.2). Of course it would be totally unnecessary to allocate the buffer in that function, so of course the global variable is the better option. I just wanted to point out that allocating a buffer and appending a string to a buffer nearly take the same time (at least when the allocation list is empty). Also note that both operations are very fast (90ms / 100000 = 900 ns) and most of the time it will be no improvement to the overall speed of the program to do so.

Regards, Mathias

By the way: Try putting the code into a function and see how the times change!
 

pragma runLim, 0
 
Buffer buf 
 
int i, start
 
string a = "123456"
string b = "678909"
string c = "abcdef"
 
start = getTickCount_()
for (i = 0; i < 100000; i++) {
    buf = create()
        delete buf
}
print "Allocating and Deleting Buffer:" (getTickCount_() - start) "\n"
 
 
buf = create()
start = getTickCount_()
for (i = 0; i < 100000; i++) {
        buf = a 
        buf += b
        buf += c
        stringOf buf
}
print "Appending Stuff To Buffer:" (getTickCount_() - start) "\n"


 


Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS

 

"at least when the allocation list is empty"

pragma runLim, 0
 
Buffer buf 
 
int i, start
 
string a = "123456"
string b = "678909"
string c = "abcdef"
 
//*****************************
void    CreateKeepLotsBuffers()
{     start = getTickCount_()
        for (i=0; i<1000; i++)
          buf = create()
        print "Allocating and lots: \t\t" (getTickCount_() - start) "\n"
}
//*****************************
void    CreateDeleteBunchBuffers()
{     start = getTickCount_()
        for (i = 0; i < 100000; i++) {
                buf = create()
                delete buf
        }
        print "Allocating and Deleting Buffer:\t" (getTickCount_() - start) "\n"
}
//*****************************
void    GetTakeFromSingleBuffer()
{     buf = create()
        start = getTickCount_()
        for (i = 0; i < 100000; i++) {
                buf = a 
                buf += b
                buf += c
                stringOf buf
        }
        print "Appending Stuff To Buffer:\t" (getTickCount_() - start) "\n"
        delete(buf)
}
 
CreateDeleteBunchBuffers()
GetTakeFromSingleBuffer()
 
CreateKeepLotsBuffers()
 
CreateDeleteBunchBuffers()
GetTakeFromSingleBuffer()


... I get these results:

 

Allocating and Deleting Buffer:    312
Appending Stuff To Buffer:      438
Allocating and lots:            0
Allocating and Deleting Buffer: 2047
Appending Stuff To Buffer:      437


The first two are consistent with what I get on my computer with your original code; I suppose my computer is slow. Curious creating takes zero time; its deleting the buffers that takes the time; perhaps "delete" inefficiently looks through a linked list.

Notice the 6.5x increase in time to allocate a buffer (2047) when 1000 have already been allocated; yet adding and extracting takes the same time.

Pre-Allocating 40,000 Buffers instead of 1000 gives these startling results:

 

 

Allocating and Deleting Buffer:    329
Appending Stuff To Buffer:      437
Allocating and lots:            22359
Allocating and Deleting Buffer: 346891
Appending Stuff To Buffer:      453


That makes creating/deleteing a "drastic" 1000x slower than when none are pre-allocated.

So I guess I need to rephrase, "creating and deleting Buffers is very slow for complicated programs, so frequently called functions should avoid doing so".

-Louie

Not often I get days like today! lol